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Abstract 

A common situation in experimental physics is to have a signal which can not be separated from a non-interfering 
background through the use of any cut. In this paper, we describe a procedure for determining, on an event-by-event 
basis, a quality factor (Q-factor) that a given event originated from the signal distribution. This Q-factor can then 
be used as an event weight in subsequent analysis procedures, allowing one to more directly access the true spectrum 
of the signal. 
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1. Introduction 

In many physics analyses, one has non-interfering 
background events which cannot be cleanly sepa- 
rated from the desired signal. This is of particular 
concern when the problem being studied is multi- 
dimensional, i.e. there are kinematic correlations in 
the signal which must be preserved. One typical 
way to handle these situations is with the side-band 
subtraction method. Distributions constructed using 
events outside the signal region are subtracted from 
those using events inside the signal region to create 
distributions in which the backgrounds have been 
removed. While this method can be effective in some 
situations, implementing it can become problematic 
if the kinematics of the background region arc dif- 
ferent than those of the signal or if tlw^ problem is 
sufficiently multi-dimensional such that binning the 
data is severely limited by statistical uncertainties. 

In this paper, we d(^scrib(^ a proc(^durc for assign- 
ing each event in a data sample a quality factor (Q- 
factor) which gives the chance that it originates from 
the signal sample. This Q-factor can then be used as 



an event weight in subsequent analysis procedures to 
gain access to the signal distribution. For example, 
the Q-factors can be used to perform event-based 
unbinned maximum likelihood fits on the data to 
extract physical observables. Eliminating the need 
to bin the data is highly desirable for the case of 
multi-dimensional problems. We note here that this 
procedure is not appropriate for situations where 
there are quantum mechanical interference effects 
between the signal and background. 

2. Quality Factor Determination 

Consider a data set composed of n total events, 
each of which is described by m coordinates ^ 
(to > 2). The coordinates can be masses, angles, 
energies, etc. Furthermore, the data set c;onsists of 
Us signal and rib background events. Both the signal 
and background distributions are functions of the 
coordinates, S'(^) and B{^) respectively. For this 
procedure, we need to know a priori the functional 
dependence (possibly with unknown parameters) of 
the signal and background distributions in terms of 
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one of the coordinates. We will refer to this coordi- 
nate as the reference coordinate and label it ^r- 

As an example, consider the case where the ref- 
erence coordinate is a mass. The functional depen- 
dence of the signal, in terms of ^j,, might be given by 
a Gaussian or Brcit-Wigncr distribution. The back- 
ground may be well represented by a polynomial. 
In both cases, there could be unknown parameters 
{e.g. the width of the Gaussian); these are permit- 
ted when using this procedure. No other a priori in- 
formation is required concerning the dependence of 
S'(^) or -B(^) on any of the other coordinates. 

The aim of this procedure is to assign each event 
a qTiality factor, or Q- factor, which gives the chance 
that it originates from the signal sample. We first 
need to define a metric for the space spanned by ^ 
(excluding ^r)- A reasonable choice is to use Ski/TZl 
where TZk is the maximum possible difference be- 
tween the coordinates of any two events in the 
sample. Using this metric, the distance between any 
two events, dij, is given as 
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where the sum is over all coordinates except ^r- 

For each event, we compute the distance to all 
other events in the data set, and retain the ric nearest 
neighbor events, including the events itself, accord- 
ing to (1). The value of ric, which varies depending 
on the analysis, is discussed below. The Uc events 
are then fit using the unbinned maximum likelihood 
method to obtain estimators for the parameters, a, 
in the probability distribution function 



F(e.,c5) 



(2) 



where Fg and Fi, describe the functional dependence 
on the reference coordinate, ^r, of the signal and 
background respectively. 

The Q-factor for each event is then calculated as 



Qi 
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where is the value of the event's reference coor- 
dinate and dj are the estimators for the parameters 
obtained from the event's fit. 

If one wants to bin the data, the signal yield in a 
bin is obtained as 



(4) 



where is the number of events in the bin. For ex- 
ample, to construct a histogram (of any dimension) 
of the signal, one would simply weight each event's 
contribution by its Q-factor. 

3. Error Estimation 

It is also important to extract the uncertainties 
on the individual Q-factors so that we can obtain 
error estimates on measurable quantities. The full 
covariance matrix obtained from each event's fit, C^, 
can be used to calculate the uncertainty in Q as 



dQ 



(5) 



When using these values to obtain errors on the 
signal yield in any bin, we must consider the fact 
that the nature of our procedure leads to highly- 
correlated results for each event and its ric near- 
est neighbors; thus, simply adding the ag values in 
quadrature would certainly underestimate the true 
error. The actual degree of correlation of the Q- 
factors would depend on the population of the bins. 
A safe choice is to assume 100% correlation; thus. 



(6) 



which provides an overestimate of the true uncer- 
tainty inherent in the procedure. 

In addition to the uncertainties associated with 
the fits, there will also be a purely statistical error 
associated with the signal yield in each bin, given 
by Poisson statistics. For large values of y, the sig- 
nal yield obtained using (4) , this can be taken to be 
vO^; however, for smaller y the upper limit is better 
approximated by 1 + y/y + 0.75 [1]. The total un- 
certainty on the signal yield in any bin is then ob- 
tain(Hl by adding the fit errors, calculated using (6), 
in quadrature with the statistical errors discussed 
above. 

4. Example Application 

As an example, we will consider the reaction 
IP — > pu; in a single (s, t) bin, i.e. a single center-of- 
mass energy and production angle bin (extending 
the example to avoid binning in production angle, 
or t, is discussed below). The co decays to ■K^TT~n° 
about 90% of the time; thus, we will assume we have 
a detector which has reconstructed 7p — > p7r+7r~7r° 
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Fig. 1. (Color Online) tf) (radians) vs cos 9: Generated decay angular distributions for all events (left), only signal events 
(middle) and only background events (right). 



events. Of course, there are production mechanisms 
other than jp — > puj which can produce this final 
state and there is no cut which can be performed to 
separate out events that originated from jp puj. 
Below we will construct a toy-model of this situation 
by generating Monte Carlo events for both signal, 
i.e. uj events, and background, i.e. non-w 7r"'"7r~7r° 
events (10,000 events were generated for each). The 
goal of our model analysis is to extract the w polar- 
ization observables known as the spin density matrix 
elements, denoted by p%iM' (discussed below) . 

In terms of the mass of the 7r+7r~7r'' system, m37r, 
the UJ events were generated according to 3-body 
phase space weighted by a Voigtian (a convolution 
of a Breit-Wigner and a Gaussian, see (12)) to ac- 
count for both the natural width of the u and de- 
tector resolution. For this example, we chose to use 
(7 = 5 MeV/c^ for the detector resolution (see Fig- 
ure 2). The goal of our analysis is to extract the 
three measurable elements of the spin density ma- 
trix (for the case where neither the beam nor target 
are polarized) traditionally chosen to be /Oqq, p\_i 
and RepiQ. These can be accessed by examining the 
distribution of the decay products (7r+7r~7r'^) of the 
Lu in its rest frame. 

For this example, we chose to work in the hclicity 
system which defines the z axis as the direction of the 
LU in the overall center-of-mass frame, the y axis as 
the normal to the production plane and the x axis is 
simply given hy x = y x z. The decay angles 9, (p are 
the polar and azimuthal angles of the normal to the 
decay plane in the w rest frame, i.e. the angles of the 
vector (pTr+ X Pt^-). The decay angular distribution 
of the Lj in its rest frame is then given by [2] 
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Fig. 2. Mass of the tt+tt'tt'' system (GeV/c^) for all 
generated events (unshaded) and for only the background 
(shaded) . 

- sin^ e COS 2(j) - V2Rep% sin 20 cos (j)) , (7) 



which follows directly from the fact that the cj is a 
vector particle; it has spin-parity — 1^ . 

We chose to use the following p^/^// values for this 
example: 



P°oo = 0.65 



0.05 



Rep% = 0.10 



(8a) 
(8b) 
(8c) 



The resulting generated decay distribution is shown 
in Figure 1. 

For the background, we chose to generate it ac- 
cording to 3-body phase space weighted by a linear 
function in m^T^ and 
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in the decay angles. Figure 2 shows the tt^tt^tt'^ 
mass spectrum for all generated events and for just 
the background. The generated decay angular distri- 
butions for all events, along with only the signal and 
background are shown in Figure 1. There is clearly 
no way to separate out the signal events through the 
use of a cut. 



4.1. Applying the Procedure 

To obtain the Q-factors, we first need to iden- 
tify the relevant coordinates, i.e. the kinematic vari- 
ables in which we need to separate signal from back- 
ground. The 7r+7r~7r° mass will be used as the ref- 
erence coordinate, = "^Stt- The stated goal of our 
analysis is to extract the P%m' elements. We will 
do this using (7); thus, only the angles 9, (j) are rel- 
evant. Other decay variables, such as the distance 
from the edge of the 7r"'"7r~7r° Dalitz plot, are not 
relevant to this analysis — though, they would be 
in other analyses (see Section 4.4). 

Usingthenotationof Section 2, ^ = (rria^, cos 0, 0) 
and the distance between any two points, dij, is 
given by 



cos Ui — cos ( 
2 
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The functional dependence of the signal and back- 
ground on the reference coordinate, Wa^, are 

Fs{m3^,a) ^ s ■V{m3.^,m^,T^,a) (11a) 

F6(to3^,(5) = 6im3„ -t- 6o, (Hb) 

where m„ = 0.78256 GeV/c^, F^ = 8.44 MeV, 
(7 = 5 MeV is the simulated detector resolution, 
d = (s, bi, bo) are unknown parameters and 
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is the convolution of a Gaussian and non-relativistic 
Breit-Wigner known as a Voigtian {'w{z) is the com- 
plex error function) . 

As stated above, the number of nearest neighbor 
events required depends on the analysis. Specifically, 
it depends on how many unknown parameters there 
are, along with the functional forms of Fg and Ft,. For 
this relatively simple case, the value ric = 100 works 
well. For each simulated event, we then find the 
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Fig. 4. (Color Online) Mass of the 7r+7r~7r'' system (GeV/c^) 
for all generated events (unshaded), only generated back- 
ground events (shaded) and all generated events weighted 
by 1 — Q (dashed-red). 

Tic closest events (containing both signal and back- 
ground) and perform an unbinned maximum likeli- 
hood fit, using the CERNLIB package MINUIT [3], 
to determine the estimators a. The Q-f actors are 
then calculated from (3) and the uncertainties are 
straightforward to calculate using Section 3. 

Figure 4 shows the comparison of the extracted 
and generated background m^^ distributions inte- 
grated over all decay angles. The agreement is quite 
good; however, we are looking for more than just 
global agreement. Figure 3 shows the extracted an- 
gular distributions for the signal and background. 
The agreement with the generated distributions is 
excellent (see Figure 1) . We can also compare the 
Q-factors extracted by the fits to the theoretical dis- 
tributions from which our data was generated. Fig- 
ure 5 shows that the extracted values are in very 
good agreement with the generated values. 

We conclude this section by discussing the impor- 
tance of quality control in the fits. For this example, 
we performed 20,000 independent fits to extract the 
Q-factors. To avoid problems which can arise due to 
fits not converging or finding local minima, each un- 
binned maximum likelihood fit was run with three 
different sets of starting values for the parameters 
a: (1) 100% signal; (2) 100% background; (3) 50% 
signal, 50% background. In all cases, the fit with the 
best likelihood was used. The ric events were then 
binned and a x^/n-df was obtained. In about 2% of 
the fits the x^/nd/ was very large, a clear indicator 
that the fit had not found the best estimators a. For 
these events, a binned fit was run to obtain the 
Q-factor. 
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Fig. 3. (Color Online) 4> (radians) vs cosd: Extracted decay angular distributions for all events (left), for events weighted by 
Q, signal (middle), and for events weighted by 1 — Q, background (right). 




Fig. 5. (Color Online) (a) Calculated Q-factors vs generated Q-factors. The red line indicates where Qcalc = Qge. 
(b) Qcalc ~ Qgen- The difference between the generated and calculated Q-factors. 



4.2. Examining the Errors 

As discussed in Section 3, the covariance matrix 
obtained from each event's fit can be used to obtain 
the uncertainty in Q, erg, using (5). The nature of 
our procedure leads to a high degree of correlation 
between neighboring event's Q-factors. This means 
that adding the uncertainties in quadrature would 
definitely underestimate the true error. In Section 3, 
we argued that a better approach was to assume 
100% correlation which provides an overestimate of 
the true error. 

To examine the error bars in our toy example, we 
chose to project our data into a one-dimensional dis- 
tribution in cos 9. This was done to avoid bin occu- 
pancy issues which arise in the two-dimensional case 
due to limitations in statistics. Figure 6(a) shows the 
comparison between the generated and calculated 
cos 9 distributions. The agreement is excellent. The 
error bars on the calculated points were obtained us- 



ing (6). For this study, we ignore the Poisson statis- 
tical uncertainty in the yield due to the fact that the 
number of generated events is known. In a real world 
analysis, these should be included in the quoted er- 
ror bars. 

We can examine the quality of the error estima- 
tion by examining the difference between the gen- 
erated and calculated yields in each bin. Ay. Fig- 
ure 6(b) shows the comparison between Ay, ay ob- 
tained assuming 100% correlation and ay obtained 
assuming no correlation, i.e. adding the individual 
uncertainties in quadrature. As expected, the corre- 
lated errors provide an overestimate of Ay in every 
bin, while the uncorrelated errors greatly underesti- 
mate A3^ in the majority of bins. 

It is not surprising that the errors obtained us- 
ing (6) are better estimates of the true uncertainty 
when the bin occupancy is closer to ric. We could 
come up with a scheme to scale the size of the er- 
ror bars according to bin occupancy; however, re- 
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Fig. 6. (Color Online) (a) Signal yield vs cos 6 for generated signal events (dashed) and all generated events weighted by 
Q-factors (blue circles). The error bars on the extracted yields were obtained using (6). (b) Comparison of the true error on 
the signal yield, A^^ = \ygen — ycalcl (open black squares), to the error bars obtained using the ctq values obtained from (5) 
combined assuming 100% correlation (blue circles) and assuming no correlation (shaded band at bottom). 



call that in this example we fit the background with 
the functional form to which it was generated. This 
typically will not be the case for real data. In a real 
analysis, fitting with different functional forms and 
examining the extracted signal distributions to es- 
timate systematic effects due to the choice of back- 
ground is generally required. In this case, one should 
simply consider that the uncorrelated error bars are 
overestimates for each choice of background func- 
tion when estimating the systematic uncertainty in 
the signal yields. 

4.3. Extracting Observables: Event-Based Fitting 
using Q- Factors 

The goal of our model analysis is to extract the 
spin density matrix elements. When studying multi- 
dimensional problems, binning the data is often un- 
desirable due to limitations in statistics. The Q- 
factors obtained for each event above can be used in 
conjunction with the unbinned maximum likelihood 
method to avoid this difficulty. 

If we could cleanly separate out the signal events 
from the background, then the likelihood function 
would be defined as 

£^]lW{e,,cj,,), (13) 

i 

where W is the decay angular distribution defined 
in (7). Wc could then obtain estimators for the spin 
density matrix elements by minimizing 



-ln£^~J2^nW{e,,cl),). (14) 

i 

In this example, it is not possible to separate the 
signal and background event samples; however, we 
can use the Q-factors to achieve the same effect by 
rewriting (14) as 

n 

-ln£^-Y,Q^^^W{0„(j),), (15) 

i 

where the sum is now over all events (signal and 
background). Thus, the Q-factors are used to weight 
each event's contribution to the likelihood. 

Using the Q-factors obtained in Section 4.1, min- 
imizing (15) yields 

p|]o = 0.654 ± 0.011 (16a) 

= 0.046 ±0.008 (16b) 

Rep% = 0.099 ± 0.007, (16c) 

where the uncertainties are purely statistical (ob- 
tained from the fit covariance matrix) . Thus, the val- 
ues extracted for the spin density matrix elements 
are in excellent agreement with the values used to 
generate the data given in (8). 

4.4. Extending the Example 

To extend this example to allow for the case where 
the data is not binned in production angle, we would 
simply need to include cos O'^j^j or t in the vector 
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of relevant coordinates, ^. To perform a full partial 

wave analysis on the data, we would also need to in- 
clude any additional kinematic variables which fac- 
tor into the partial wave amplitudes, e.g. the dis- 
tance from the edge of the tt+tt^tt'' Dalitz plot (typ- 
ically included in the lo decay amplitude) . We would 
then construct the likelihood from the partial waves 
and minimize — In £ using the Q-factors obtained 
by applying our procedure including the additional 
coordinates. An example of this can be found in [4] . 

5. Conclusions 

In this paper, we have presented a procedure for 
separating signals from non-interfering backgrounds 
by determining, on an event-by-event basis, a quality 
fac;tor (Q-factor) that a given event originated from 
the signal distribution. We have shown that this Q- 
factor can be used as an event weight in subsequent 
analysis procedures to allow more direct access to 
the true signal distribution. Though this procedure 
may be computationally expensive, in principle it 
only needs to be performed once for each event in 
the data sample. 
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